Ramp function

 ( Real Analysis ) 

The ramp function is a unary function real function, whose graph is shaped like a ramp. It can be expressed by numerous definitions, for example "0 for negative inputs, output equals input for non-negative inputs". The term "ramp" can also be used for other functions obtained by scaling and shifting, and the function in this article is the unit ramp function (slope 1, starting at 0).

In mathematics, the ramp function is also known as the positive part.

In machine learning, it is commonly known as a ReLU activation function or a rectifier in analogy to half-wave rectification in electrical engineering. In statistics (when used as a likelihood function) it is known as a tobit model.

This function has numerous applications in mathematics and engineering, and goes by various names, depending on the context. There are differentiable variants of the ramp function.

---

Definitions The ramp function () may be defined analytically in several ways. Possible definitions are:

• A piecewise function: $$R(x)\; :=\; \backslash begin\{cases\}$$

x, & x \ge 0; \\ 0, & x<0 \end{cases}

• Using the Iverson bracket notation: $$R(x)\; :=\; x\; \backslash cdot\; x$$ or $$R(x)\; :=\; x\; \backslash cdot\; x$$
• The max function: $$R(x)\; :=\; \backslash max(x,0)$$
• The arithmetic mean of an independent variable and its absolute value (a straight line with unity gradient and its modulus): $$R(x)\; :=\; \backslash frac\{x+|x|\}\{2\}$$ this can be derived by noting the following definition of , $$\backslash max(a,b)\; =\; \backslash frac\{a\; +\; b\; +\; |a\; -\; b|\}\{2\}$$ for which and
• The Heaviside step function multiplied by a straight line with unity gradient: $$R\backslash left(\; x\; \backslash right)\; :=\; x\; H(x)$$
• The convolution of the Heaviside step function with itself: $$R\backslash left(\; x\; \backslash right)\; :=\; H(x)\; *\; H(x)$$
• The integral of the Heaviside step function: $$R(x)\; :=\; \backslash int\_\{-\backslash infty\}^\{x\}\; H(\backslash xi)\backslash ,d\backslash xi$$
• Macaulay brackets: $$R(x)\; :=\; \backslash langle\; x\backslash rangle$$
• The positive part of the identity function: $$R\; :=\; \backslash operatorname\{id\}^+$$
• As a limit function: $$R\backslash left(\; x\; \backslash right)\; :=\; \backslash lim\_\{a\backslash to\; \backslash infty\}\; \backslash begin\{cases\}\; \backslash frac\{1\}\{a\}\; ,\backslash quad\; x=0\; \backslash \backslash \; \backslash dfrac\{x\}\{1-e^\{-ax\}\},\backslash quad\; x\backslash neq\; 0\backslash end\{cases\}$$

It could approximated as close as desired by choosing an increasing positive value $a>0$.

---

Applications The ramp function has numerous applications in engineering, such as in the theory of digital signal processing.

In finance, the payoff of a call option is a ramp (shifted by strike price). Horizontally flipping a ramp yields a put option, while vertically flipping (taking the negative) corresponds to selling or being "short" an option. In finance, the shape is widely called a "hockey stick", due to the shape being similar to an ice hockey stick.

In statistics, hinge functions of multivariate adaptive regression splines (MARS) are ramps, and are used to build .

---

Analytic properties

---

Non-negativity In the whole domain the function is non-negative, so its absolute value is itself, i.e. $$\backslash forall\; x\; \backslash in\; \backslash Reals:\; R(x)\; \backslash geq\; 0$$ and $$\backslash left|\; R\; (x)\; \backslash right|\; =\; R(x)$$

---

Derivative Its derivative is the Heaviside step function: $$R\text{'}(x)\; =\; H(x)\backslash quad\; \backslash mbox\{for\; \}\; x\; \backslash ne\; 0.$$

---

Second derivative The ramp function satisfies the differential equation: $$\backslash frac\{d^2\}\{dx^2\}\; R(x\; -\; x\_0)\; =\; \backslash delta(x\; -\; x\_0),$$ where is the Dirac delta. This means that is a Green's function for the second derivative operator. Thus, any function, , with an integrable second derivative, , will satisfy the equation:

---

Fourier transform $$\backslash mathcal\{F\}\backslash big\backslash \{\; R(x)\; \backslash big\backslash \}(f)\; =\; \backslash int\_\{-\backslash infty\}^\{\backslash infty\}\; R(x)\; e^\{-2\backslash pi\; ifx\}\; \backslash ,\; dx\; =\; \backslash frac\{i\backslash delta\; \text{'}(f)\}\{4\backslash pi\}-\backslash frac\{1\}\{4\; \backslash pi^2\; f^2\},$$ where is the Dirac delta (in this formula, its derivative appears).

---

Laplace transform The single-sided Laplace transform of is given as follows, $$\backslash mathcal\{L\}\backslash big\backslash \{R(x)\backslash big\backslash \}\; (s)\; =\; \backslash int\_\{0\}^\{\backslash infty\}\; e^\{-sx\}R(x)dx\; =\; \backslash frac\{1\}\{s^2\}.$$

---

Algebraic properties

---

Iteration invariance Every iterated function of the ramp mapping is itself, as $$R\; \backslash big(\; R(x)\; \backslash big)\; =\; R(x)\; .$$

---

See also

• Tobit model
• Rectifier (neural networks)

Post Comment